Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup to prepare for using mmap pointer in external memory. #9317

Merged
merged 3 commits into from
Jun 21, 2023

Conversation

trivialfis
Copy link
Member

@trivialfis trivialfis commented Jun 19, 2023

Extracted from #9315 .

  • Update SparseDMatrix comment.
  • Use a pointer in the bitfield. We will replace the std::vector<bool> in ColumnMatrix with bitfield.
  • Clean up the page source. The timer is removed as it's not accurate once we swap the mmap pointer into the page.

src/common/bitfield.h Show resolved Hide resolved
@@ -70,8 +73,9 @@ struct BitFieldContainer {
};

private:
common::Span<value_type> bits_;
static_assert(!std::is_signed<VT>::value, "Must use unsiged type as underlying storage.");
value_type* bits_{nullptr};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain why we should use a raw pointer here? You mentioned the use of std::vector<bool> in ColumnMatrix. Does the span behave badly when value_type == bool ?

Copy link
Member Author

@trivialfis trivialfis Jun 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's used in hot loops, the check used in span can be expensive.

We won't be using std::vector<bool> after #9315 . In that PR, I have defined a reference counted view that's a replacement for std::vector, with the ability that we can swap out the underlying storage when needed. It doesn't have the bool specialization, therefore, an additional bitfield is used.

@trivialfis trivialfis merged commit 54da4b3 into dmlc:master Jun 21, 2023
@trivialfis trivialfis deleted the ext-cleanups branch June 21, 2023 22:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants